Experiment with concurrency levels by adespawn · Pull Request #425 · scylladb/nodejs-rs-driver

adespawn · 2026-03-30T13:04:49Z

Refs: #73

With planned changes that touch the concurrency, we would want to see the driver performance across different concurrency levels.

This PR introduces a refactor to the current benchmarks, which would allow us to add benchmarks parameterised on the concurrency level easily, and adds those benchmarks.

I would recommend reviewing the first commit in vscode (or similar), to better see what changes and what's kept unchanged

Introduces changes that were generated with LLM.

Example results [with not yet published PR] (x axis is the concurrency level on the benchmark) - I need to add beter labeling support to the benchmarking tool)

Copilot

Pull request overview

Refactors the JS benchmark harness to support easily parameterizing concurrency level (for planned concurrency-related driver changes) and adds new “fixed query count / variable concurrency” benchmark variants.

Changes:

Introduces a centralized JS benchmark entrypoint (benchmark/logic/benchmark.js) and updates existing benchmark modules to export a common function signature.
Updates benchmarker runner configs and wrapper scripts to call the centralized entrypoint (benchmark names no longer include .js).
Adds new *_fixed benchmark variants in runner configs to sweep concurrency levels while keeping query count fixed.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
benchmark/runner-config/scylladb-driver/config.yml	Switches backend commands to the centralized benchmark entrypoint; adds `*_fixed` variants.
benchmark/runner-config/cassandra-driver/config.yml	Same as above for the cassandra-driver backend; adds `*_fixed` variants.
benchmark/runner-config/run-js-benchmark.sh	Wrapper now calls `benchmark/logic/benchmark.js` with `<driver> <benchmark-name> <N>`.
benchmark/runner-config/run-js-benchmark-fixed.sh	New wrapper for concurrency-sweep scenarios (step controls concurrency).
benchmark/runner-config/config.yml	Adds benchmark definitions for `*_fixed` variants and adjusts default run count.
benchmark/logic/benchmark.js	New centralized dispatcher that loads the driver, creates a client, and invokes the selected benchmark module.
benchmark/logic/insert.js	Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_insert.js	Converted to exported benchmark function; now accepts `concurrencyLevel`.
benchmark/logic/select.js	Converted to exported benchmark function delegating to `parametrized_select`.
benchmark/logic/parametrized_select.js	Refactored to accept `(cassandra, client, rowCount, stepCount)` rather than argv.
benchmark/logic/large_select.js	Converted to exported benchmark function delegating to `parametrized_select`.
benchmark/logic/paging.js	Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_paging.js	Converted to exported benchmark function; now takes runner-provided `concurrencyLevel`.
benchmark/logic/batch.js	Converted to exported benchmark function; clarifies batch-size constant naming/comment.
benchmark/logic/deser.js	Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_deser.js	Converted to exported benchmark function; now accepts `concurrencyLevel`.
benchmark/logic/ser.js	Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_ser.js	Converted to exported benchmark function; now accepts `concurrencyLevel`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

benchmark/logic/benchmark.js

benchmark/logic/insert.js

benchmark/logic/concurrent_paging.js

nikagra · 2026-04-01T13:59:22Z

The default stepCount values vary widely across modules (e.g. batch.js: 3,000,000 vs concurrent_paging.js: 1,280) with no explanation. Since run-js-benchmark-fixed.sh always uses these defaults, it's worth adding a comment in each module clarifying the intended scale.

benchmark/runner-config/run-js-benchmark-fixed.sh

benchmark/runner-config/config.yml

benchmark/runner-config/run-js-benchmark-fixed.sh

benchmark/runner-config/run-js-concurrency-benchmark.sh

adespawn · 2026-04-02T09:47:55Z

The default stepCount values vary widely across modules

Those values were taken from the old python runner, and were determined to be the sweet spot, where the benchmark takes around 1-2 minutes to execute

wprzytula · 2026-04-02T12:17:03Z

benchmark/runner-config/cassandra-driver/config.yml

♻️ Can we extract the refactor that changes the invocation from using insert.js to insert to a separate commit?

From what other changes that are present in that commit?

benchmark/runner-config/config.yml

wprzytula · 2026-04-02T12:18:23Z

benchmark/logic/batch.js

-const iterCnt = parseInt(process.argv[3]);
-// Expectantly determined max batch size, that doesn't cause database error.
-const step = 3971;
+module.exports = function (cassandra, client, stepCount, _concurrencyLevel) {


⛏️ Here you use underscore in _concurrencyLevel, and elsewhere not.

In non concurrent benchmarks I ignore this parameter. For this reason I use underscore to indicate that

benchmark/logic/benchmark.js

wprzytula · 2026-04-02T12:23:17Z

benchmark/logic/benchmark.js

+const stepCount = process.argv[4] !== "default" ? parseInt(process.argv[4], 10) : undefined;
+const concurrencyLevel =process.argv[5] !== "default" ? parseInt(process.argv[5], 10) : 100;


♻️ Let's have separate constants: defaultConcurrencyLevel, defaultStepCount and advise to adjust them as needed. Then, formulas in concurrencyLevel will not have to be ever touched.

While defaultConcurrencyLevel makes sense, I'm not sure why you want defaultStepCount to be defined as constants.

benchmark/runner-config/run-js-benchmark-fixed.sh

benchmark/runner-config/cassandra-driver/config.yml

This commit moves the entry points for individual benchmarks to separate files. This change will allows us to easily create new benchmarks, that are parametrized on concurrency level, without the need to create a separate copy of the files.

This adds a new version of the existing benchmarks, that try to parameterize the concurrency level, rather then the number of iterations.

This benchmark was still using it's own row check count, instead of the one from utils

adespawn requested a review from Copilot March 30, 2026 13:04

adespawn self-assigned this Mar 30, 2026

adespawn added the benchmark Things related to benchmarking the driver label Mar 30, 2026

adespawn added this to the GA milestone Mar 30, 2026

Copilot started reviewing on behalf of adespawn March 30, 2026 13:05 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

benchmark/logic/benchmark.js Outdated Show resolved Hide resolved

benchmark/logic/insert.js Show resolved Hide resolved

benchmark/logic/concurrent_paging.js Outdated Show resolved Hide resolved

benchmark/logic/concurrent_paging.js Show resolved Hide resolved

adespawn force-pushed the experiment-with-concurrency-levels branch from 225088c to ea74b6c Compare March 30, 2026 13:27

adespawn requested review from nikagra and wprzytula March 30, 2026 13:27

nikagra reviewed Apr 1, 2026

View reviewed changes

adespawn force-pushed the experiment-with-concurrency-levels branch from ea74b6c to ae4385c Compare April 2, 2026 09:56

adespawn modified the milestones: GA, 0.5.0 Apr 2, 2026

adespawn force-pushed the experiment-with-concurrency-levels branch from ae4385c to 57fc30b Compare April 2, 2026 11:57

adespawn requested a review from nikagra April 2, 2026 11:57

adespawn force-pushed the experiment-with-concurrency-levels branch 2 times, most recently from ffcca89 to 1e687bc Compare April 2, 2026 12:28

wprzytula requested changes Apr 2, 2026

View reviewed changes

adespawn force-pushed the experiment-with-concurrency-levels branch 4 times, most recently from fca37ed to 593c31b Compare April 9, 2026 08:50

adespawn added 3 commits April 9, 2026 14:53

Refactor benchmarking

fd4bd84

This commit moves the entry points for individual benchmarks to separate files. This change will allows us to easily create new benchmarks, that are parametrized on concurrency level, without the need to create a separate copy of the files.

Add benchmark that are parametrized on concurrency

221c343

This adds a new version of the existing benchmarks, that try to parameterize the concurrency level, rather then the number of iterations.

Update batch benchmark to use common utils

a3a795c

This benchmark was still using it's own row check count, instead of the one from utils

adespawn force-pushed the experiment-with-concurrency-levels branch from c621a25 to a3a795c Compare April 9, 2026 12:53

Add rust version of concurrency parameterized benchmarks

b90d2fd

		const stepCount = process.argv[4] !== "default" ? parseInt(process.argv[4], 10) : undefined;
		const concurrencyLevel =process.argv[5] !== "default" ? parseInt(process.argv[5], 10) : 100;

Conversation

adespawn commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nikagra commented Apr 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adespawn commented Apr 2, 2026

Uh oh!

wprzytula Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

adespawn Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wprzytula Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

adespawn Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wprzytula Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

adespawn Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

adespawn commented Mar 30, 2026 •

edited

Loading